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(5^1) Title: IMPROVED METHOD AND APPARATUS FOR CREATING VIRTUAL WORLDS 


(57) Abstract 



A computer display displays a data flow network as a plurality of interconnected units. An interactive program allows the 
appearance of the plurality of interconneaed units to be changed. The data flow network then is automatically altered to corre- 
spond with the visual changes. 


FOR THE PURPOSES OF INFORMATION ONLY 


Codes used to identify States party to the PCX on the front pages of pamphlets publishing international 
applications under the PCT. 


AT 

Austria 

es 

Spaiji 

MC 

Madagascar 

AU 

Ausualia 

PI 

Finland 

ML 

Mali 

BB 

Bjrbsdd 

pa 

France 

MN 

Mongiolia 

BE 

Bcigtum 

CA 

Cuhon 

MQ 

Mourttania 

BF 

Burkina Fa^o 

CD 

Untied Kingiloffl 

MW 

Matocfi 

BG 

Bulgaria 

CN 

Guinea 

NL 

Netherlands 

8J 

Benin 

Gft 

Greece 

NO 

Norway 

BR 

Bra/JI 

HU 

Huntiary 

PL 

Poland 

CA 


IT 

Italy 

ao 

Romania 

CP 

(Antral Afrknn Republic 

JP 

Japan 

so 

Sudan 

CC 

Congo 

KP 

Oemocraiic People's Republic 

ss 

Svveden 

CH 

Swiuertami 


of Korea 

SN 

Senc^l 

CI 

Cote d'lvoin; 


Republic of Korea 


Sotrtet Union 

CM 

Cameroon 

LI 

Ueehienueio 

TO 

Ched 

CS 

Cecchosiovakij 

LK 

an Lantia 

TC 

Togo 

DE« 

Germany 

LU 

Luxembourg 

US 

United States of America 


DenmartE 

MC 

Monaco 




+ Any designation of '^SU** has effect in the Russian Federation. It is noa yet Einctvn whether 
any such designati n has effect in ther States of the former Soviet UnioQ. 


wo 92/09948 PCr/US9 1/08948 


IMPROVED METHOD AND APPARATUS FOR CREATING VIRTUAL WQPT ^nc 

5 BACKGROUND OF THE INVENTION 

This invention relates to virtual reality systems 
and, more particularly, to a method and apparatus for 
creating a virtual world. 

Users of computer systems are now able to create 
10 virtual realities which they may view and interact with. 
One type of virtual reality system is disclosed in U.S. 
patent application No. 535,253, filed June 7, 1990, entitled 
"Virtual Reality Network," the disclosure of which is 
incorporated herein by reference. One task which must be 
15 performed is the creation of the virtual worlds within which 
the users interact. This can be a very tedious task, since 
a complete description of all virtual objects in the virtual 
world, including their constraints of motion, hierarchy, 
color, texture and other attributes must be made and entered 
20 into the virtual reality computer. Thus, it is desirable to 
make virtual world creation as simple as possible. 

One product used to create virtual worlds is the 
"Body Electric" program discussed in the above-identified 
patent application. The Body Electric program allows the 
25 user to create virtual world hierarchy trees and to couple 

data (real world or synthesized) to the tree nodes. Further 
details of the program maybe found in Appendix 1. 

SUMMARY OF THE INVENTION 
The present invention is directed to an improved 
method and apparatus for creating virtual worlds wherein 
users may manipulate pictorial objects on a computer screen 
to effect complicated operations on the virtual world 
database. In one embodiment of the present invention, a 
35 computer display displays a data flow network as a plurality 
of interconnected units. An interactive program allows the 
appearance of the plurality of interconnected units to be 
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changed. The data flow network then is automatically 
altered to correspond with the visual changes. 

BRIEF DESCRIPTION OF THE DRAWINGS 
5 Fig. 1 is a diagram of a display screen which 

incorporates features according to the present invention; 
and 

Fig. 2 is a diagram of a display screen which 
incorporates interprocessor communication features according 
10 to the present invention • 

BRIEF DESCRIPTION OF THE APPENDICES 
Appendix 1 is a text description of the operation 
of a program that is used for creating virtual world 
15 databases ; 

Appendix 2 is a listing of numerous examples of 
virtual worlds created with the present invention; 

Appendix 3 is a source code listing of "Body 
Electric" software which incorporates features of the 
20 present invention; and 

Appendix 4 is a source code listing of an 
interprocessor editor according to the present invention. 

DESCRIPTION OF SPECIFIC EMBODIMENTS 
25 Fig. 1 is a diagram showing a computer display 

showing features of the present invention • It should be 
referred to in the discussion which follows. 
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The Names Window 


The introduction of (he Names window is one of the nx>5t fundamental 
changes of Body Electric 17b3. This window replaces many of the 
functions that the DM dialog box originally peif onned. 

It might be useful to have Body Electric lunniog while reading this. Just 
start up BE 2.7b3 and open a *New* file (from the QlC menu). Body 
10 Elecmc will ask you if you warn to create a Tree or DM file. Choose DM. 


The *nryoe" List. 


15 


Below is the Names Window with the 'type' box selected. This window 
is invoked by selecting "Names'* from the Window menu. 


20 


25 


30 
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MDm Names SEl 


raw I tfm [alpha 


niseel laneous 
PqssBow 

Library 
Scait/Offsat 
ShovUa I uft 
Const 

Run Script 
Latch 
Svltch 

Singta Gesture 

Tabfs Lookup 

Sort Ttto hfumbtrs 

BytftSwop 

Clock 

Uaricrock 
Filters 

LfmltTo 

St«pByX 

Oftbounce 

Str«tchPuls« 

Saooth 

Deloy 
Arithmetic 

Plus 

Minus 

Times 

Divide 

Hodu I o 

fliiflnd 


25 


The names you wiU see in this list axe the DM types that can be used In 
order to aeaie a DM of any of these types, simply click on the name of the 
DM type yon desire (without releasing the mouse button) and drag the DM 
onto a seleaed DM file window. Note that the DM file window ^ould be 
selected (and not the Names window). 
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The l^aw" List 


raw ESESIIIJH 
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A listing of available raw inputs will appear when this box is checked. A 
raw input can be connected to a DM in a seleaed DM window by doing 
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basically the same process as was done to drag the DM out. Click down 
on the raw input you need and drag it dose to the DM input (on the left 
side of the DM) that you wish to connect it to. When the raw input is 
within connecting distance, the input box it is closest to will light up. By 
releasing the mouse button, the raw input should connect itself to the DM. 
Another method of connecting a raw input to a DM is to click down on the 
DM input and then setea the raw input needed. When the mouse button is 
releaseidt the connection should be made. 


The 'Tree" Lirt 



pitch 


alpha 


roll 


When the tree box is selected, a listing of the tree nodes from the Swivel 
file that is activated will appear. The tree file below is from one of VPL*s 
worlds named "Kitchen**, tree node inputs and outputs can be conneaed 
to DM*s in the same fashion as raw inputs are. One thing thai is slightly 
different is that when using tree nodes as inputs or outputs, the user must 
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20 


specify which of the six degrees of freedom is being used. These six 
degrees of freedom are x, y, z, yaw, pitch, and rolL (sec also 'The 
Lower Six Options.") 


Names ^^Rs 


tree CgSlBB 


M>v jpitch 


roll 


CtockUoll 
ClockFocs 

Sound2 

90Clock 

60Crock 

30Ctock 

ClockTvtt(vtt 1 
C(ockTwelve2 

S«con^and 

ninuteHond 

HourHond 
Counterlop 

Coun torfiaseboard 

Sink 
HotGrob 

Ho IGrob^ounds. 
HolCrab^ln. 
HolGrobuiox. 
Co I dCrab 

Co I dGr Qb^ounds^ 
Co I dGr abub in. 
Co I dOrab. 
Fauc«t 

FlovingUaler 

Sound4 
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The "fllDhfl" SplPctinn 

Name! 

dm Pf^g^ 
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When checked, this box makes the selected listing ^)pear in alphabetical 
order. Sometimes this is advantageous. When loolang for a specific DM 
it can be very useful to be able to find it alphabetically. Othertimesan 
alpbabedcal listing can be confusing. For example, if you were trying to 
see the tree structure of a world, alpha would prevent you firom seeing this 
structure. 


35 
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The "dm*^ IMu? 


I Xrtt type | 


14 15 16 


10 


Highlighting the dm box will show a list of the DM*s that are in the active 
DM file. When you create a DM and name it , it will appear in this menu. 
With 'dm' comes the numbers 1 through 6 in the lower half of the names 
menu section (see above). 

Tlie dm nnenu item is useful as a time saver* Instead of searching through 
your file foraDM, you can find it here and use it as an input or output 
Just selea the in or out port (1*6) that you wish to use and then use this 
list like you would use the •'raw" and "tree^lists. 


The Lower iSix Ontions 


15 


J.. 

z 


3 

pitch 

roll 

14 15 

6 
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The six bottom boxes in the Names Window menu are modifiers. 
Modifiers appear in three cases: with die Tree, Type, and DM lists. These 
modifiers specify either the degree of freedom or the number of the input 
depending on the list chosen. For example, when creating a set of DM's 
that will move an object in VR, Body Electric usually needs to know what 
axis of the object should move. Do you want the object to move along the 
X axis, or do you wont it to roll? In order to use the coxrtct "axis'* of an 
objea, selea x, y, z, yaw, pitch, or roll before **grabbing" the desired 
node. In the same way, choose the number of the input or output you are 
using. 


25 


The "Flex" Window 


30 


Body Electric 2Jb3 now allows the user to have more than one window 
open at a time. Because of this, the menu items "Flex" and "Tree" firom 
the Window's menu no longer exisL In place of this. Body Elecoic now 
lists the names of the files that are open. To open a DM file (what used to 
be opened by •'Flex'' in the V^dows menu) called name, choose 
mime^DM. To open a tree file (or the Swivel file), choose name. 


35 
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AHivafg fWinHnws Mt»nu^ 


iUindoujs 


Stock Ulini 
Tile iUfndo 
Close All U 
Rctluate 
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15 


Bcxiy Electric must know which tiee and DM file to run at any point in 
time. Because multiple files may be open simultaneously, the user must 
alert Body Electric to which set of files are being used. That is, the user 
must make sure that the files being worked on are "Activated". To do 
this, open the window to be activated. With this window selected, choose 
"Activate" fiora the Ayindows menu. 

Li order to check whether i file is activated or not, look in the Windows 
menu. Files that are acdve have solid diamonds to the left of their name. 
Libraries and Nested DM's that belong to activated files have hollow 
diamonds next to their names. 


BE Wflrht?encfi Tools 
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This is a picnire of a new DM window. On the left hand side of the DM 
window are 6 tools. From top to bottom they are named the Selea tool, 
the Hand tool, the On/Off Switch tool, the Calibrate tool (it's a wrench), 
the Input/Output tool, and the DM description tool 


The Select tool is the default tool for DM windows. It operates in the 
same fashion as the standard Macintosh arrow. To select a single box, 
click on it To select several DM*s in the same area, pull a box out around 
the desired boxes. To selea more than one DM one by one. hold down 
the shift buaon while clicking on the desired DM*s. 
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With the Hand tool, the user can "grab'* the woricspacc within the DM 
window and slide it around like a sheet of paper on a desk. This is an 
alternative to using the scroll bars. 


The On/OIT Switch tool allows the user to turn DM*s on or off very 
quickly. There are two ways of using this tooL One way is to select the 
On/Off tool and then click on the DM that needs to be turned on or off. 
Another way allows tlte user to turn on or off multiple DM*s 
simultaneously. Fust select the DM's you want to change (see the Selea 
Tool). Then hold down the mouse buaon on the On/Off tool until it starts 
blinking. When you release the mouse button, the selected DM's will 
either be turned on or off. 


The Scissors tool, is used to break connections that already exist 
between DM's and other DMs, Tree or Raw inputs. To use this tool 
selea it and then click on the input pad you want to disconnect The input 
that's going into that tab will be disconnected and the tab will go to its 
default if it has a defauk or if not the DM will be uimed off. 


The Wrench, or Calibrate tool, is used to set the internal informadon of 
a DM. For example, when calibrated, the Scale/Offset DM presents the 
dialog box below. Not all DM's have calibration boxes (e.g., Srunes, 
Average Two, Cube) 


FfKed ScQle/Offset OM tnoduie *UPL 
Reseerch'. 


Rnd the neiu offset uolue: 


0.00000 


[cancel] 


35 


The Input/Output Tool lets the user modify the number and type of 
inputs and outputs for each DM. To use it, select this tool and click on 
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10 


any DM. Here is what a common Input/Output dialog box looks like (this 
is from the DM Scalo/Offset). 


10 


20 


25 


Scale/Offset H DMLInes □ Display 

□ On 


Name: 


undefined 

1 

Integer 

1 














outX « ifiX ♦ const 5c4l« + const offset. 


OK 1 [ Cancel 


30 


As you can see, this is also the dialog box in which you can name a DM. 
With the DMLines box checked lines are drawn from the inputs of diis 
DM to the outputs of other DM's. This is the default 

The Display check box 

When the On box is checked, the DM is on (i.^., it is being looked at by 
Body Electric). 

The twelve boxes O'ust below the name box) control how many inputs and 
the outputs the DM wUl have, as well as what type each input and output 
will be. Say for example that you want to have ^ inputs on a Scale/Offset 


35 
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15 


20 


DM of type Integer. To do this, click down on the third box down in the 
left most column. The following menu will appear. 


UDUSBd 

Integer 

Long 

FiKed 

Real 

Point 

S^me 

PolntLUt 

undefined 


By selecting '^teger'* and leleasine the mouse button, the integer inputs 
will now appear both in the dialog box as well as on the DM itself. The 
same method can be used to change the type and number of the outputs. 

Note that by dianging the inputs of a DM, the outputs usually are altexed 
automatically based upon the DM being woiked on. 

And finally, the bottom-most box contains a description of the DM. 


By selecting the DM Description Tool and cliddns on a DM, a 
description of the DM will appear across the top of tte window. This box 
describes what the DM does. It is useful for fimling out what a DM does 
as well as for double-checking what any q)edfic input is used for. 


25 
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Another of the fundamental changes in Body Electric is the addition of 
Libraries and Nested DM's. 


Nested DM*s are DM networks that exists inside other DM networks. 
When the outer networic executes, it calls the Nested DM which executes 
it's network, and if it in turn has Nested DM's inside it they are called etc. 
This tiierarchical descent occuis to as niany levels as necessary. Each 
Nested DM has one input which determines if the Nested DM will be fired 
at ail and up to 5 other inputs which are passed to the DM network inside 
the Nested DM. The Nested DM can have up to 6 outputs that are passed 
from the network to the Nested DM. 


The Library DM is a special case of a Nested DM. In this case the DM 
netwoik exists inside another DM file. Tliis allows you to have fiinctions 
that are shared aitiongst different worlds. They can be added to any world 
by creating a Library DM that refers to the desired library file. This DM 
was creat^ to allow the user to include conunon and/or general Data 
Massage Modules into her or his program without too much grief. This 
DM has been oeaied in the spixh of standard programming libraries such 
as the ones in C. 


Additional Features 


Find Dialog Box; This dialog box has a new checkbox called 'Global 
Search.* When it is unchecked all searches will be in the Tree or DM 
window you are currently editing. When it is checked the search will be 
conducted over all the currently opened files and for DM*s all the nested 
DM's in the files. 


Mouse Active; There is a new menu item called Mouse Active under 
the DA menu. By default this comes up checked. When you uncheck it 
the mouse and keyboard items in the Raw Data Array become inactive. 
That way you can use them as inputs to a DM network for testing. When 
you want to work on the network uncheck Mouse Active and your 
network won't react to mouse movements as you are editing. 
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Swivel IPC Specifications 
► Brief Description 

► The Swivel IPC (Inter -Process Comuni cations) has been 

designed to allow external editors access to Swivel3D worlds 
5 while Swiv 1 is running. With it we have the ability to come 

up with new ways of cr ating and editing 3D solid objects. 
Swivel can always draw any object as a list of polygons. But 
to edit an object Swivel looks at the objects type. If it is 
a standard swivel lathe object swivel calls it's internal 
editor but if it is another object type swivel tries to 
lamch the associated editor and passes a message to edit the 
given object. When you launch Swivel under Multifinder it 
runs as one application (program) , Under MultiFinder there 
' a number of applications launched at the same time. 

1^ When the user tells swivel to edit an object which uses 

another editor swivel looks up the name of the editor 
application and tries to launch that. If successful swivel 
posts a message in the SwivlPC for that editor which tells 
the editor to edit the object. The editor edits the object 
and then posts a message back to swivel to check the object 
back in. Swivel reads the object back in and updates it's 
database of objects. 

► The way messages are passed is through a resource called 
-5- 'YPJK', 0 which is in the system file and is accessible to 

all applications. This resource doesn't ftormally exist so 
the first time SwivlPC is used the resource has to be added 
to the system. 

► Since any application can access the message passing resource 
-.p, ^® wanted a way to protect our rights to the technology. 

Swxvel will ignore messages from any editor until the editor 
has "registered". This gets reset each time swivel runs. 
"Registering" means that the editor has to send a register 
message with a pointer to the string "Unauthorized use is 
subject to legal action. Until the editor does this swivel 
will ignore any messages from the editor, 
► Message passing scenario 

► User double-clicks on a object that is edited by the terrain 
editor (#67) . 

► Swivel tries to find and launch 'Terrain Editor' . Swivel 
also post a message to number 67 saying edit the object at 
location $xxxxx. 

► The Terrain Editor starts running and looks in the system 
resource for messages. It sees that there is a message from 
swivel to it to edit an object. It sends the registration 
message back to swivel and brings up a window so the user can 
edit the object. 

► When the user is done. and has clicked OK the terrain editor 
sends a message back to swivel to check in the object the 
user has edited. 

► Swivel receives the message and replaces the old copy of the 
object with the newly edited object. 

► The terrain editor can at this point either quit or hang 
around waiting for some more messages from swivel 

> Possible Messages 

► MsCheckIn 

► Check an object into swivel that has been edited by an 
editor. 

► Data, DataLen - Handle to and length of memory in the 
editor for the new object. 

*• modifiers: 

► MsCheckOut 

► MsEditObj 

► MsGetCmtObj 


40 


45 
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> MsSetCrntObj 
5 > MsDoCommand 

> MsQuiting 

> MsStairtFast 

> MsBndFast 

> MsRegisterEd 
10 > message format 

> 'C structure definition for System -Resource 'YPJK'.O. 

> typedef struct { ^ 

short whomto; /' whom is the message for */ ^ 
short whomfnn; ^ /' whom is the message from / 
15 short status; /* IPC status. Sending, received or 

error. / ^ 

short mgsscount; / Unique incrementing message 
number / 
short what; 

20 short modifier^- /* Modifier to the what field. '/ 

long when; TickCount when the message was sent, / 

long Date; / Used by Individual messages however they 
want. */ 
long DataLen; 

25 } SwivlPC, *SwivlPCPtr, **SwivlPCHandle; 

> whomto is the number of the editor or swivel the message is 
for. This is an arbitrary 16 bit number which is unique for 
each editor. Swivel is #1. 

> The only other unique address is -1. This is a broadcast 
30 to all editors. Each editor should read the message but 

not change the status. That wal all the current editors 
can see the message. Broadcast messages will last for 10 
seconds. Then anyone reading the message must set it's 
status to 0 thereby stopping the message. 
35 > whomfrm is the same as whomto but it's whom the message is 

from. 

> status - The current state of the message passing mechanism. 
It is either: 

> 0 - not being used 

40 > 1 - message has been sent and not received. 

> Any other number - The sendee has posted an error. Using 
the macintosh error conventions the errors posted should 
be negative but any number other than 0 or l is considered 
an error. 

45 > messcount - Used to keep track of the number of messages 

posted. Each time a message is posted this value is 
incremented. 

> what - The message verb. See above. 

> modifier - meaning dependant on the what field. 

50 > when - When the message was posted in Macintpsh System Ticks. 

> Data - meaning dependent on the what field. 

> DataLen - meaning dependent on the waht field. 
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The purpose of the Swivel PC is to allow the Swivel 
interface to be customized and extended for specific purposes . 
Many different editor extensions are possible. The one 
implemented in the code provided in Appendix 4 is called the 
5 Orientation Editor. 

The Orientation Editor allows those aspects of the 
Swivel file used by Body Electric for the RB2 Virtual Reality 
System to be edited and stored within the Swivel file. These 
aspects include point attributes like bound, cube, line, 
10 LineDown, or Line Right, which determine how the point is 
rendered in Body Electric, as well as enabling particular 
degrees of freedom (Roll, Pitch, Yaw, x, Y or Z) to be output 
serially to another [host] computer. The Orientation Editor 
also allows the user to send command scripts to Swivel, to 
15 check all the names of objects in the Swivel file for 

uniqueness, to inherit selected attributes, and to mark, and 
return to world viewpoints. These capabilities enhance the 
user's ability to create Swivel Models for use in RB2 . 

TRhe improved Body Electric software described may 
20 be used for creating a number of virtual worlds. Examples of 
such virtual worlds are shown in Appendix 2. 

While the above is a complete description of a 
preferred embodiment of the present invention, various 
modifications may be employed. Consequently, the scope of the 
25 invention should not be limited except as described in the 
claims. 
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WHAT IS CLAIMED IS : 

1. An apparatus for creating virtual worlds 
comprising: 

display means for displaying a data flow network 
as a plurality of interconnected units; 

means for altering the appearance of the plurality 
of interconnected units on the display; and 

means for automatically altering the data flow 
network to correspond to the altered appearance of the 
plurality of interconnected units. 


5 


10 


0 


wo 92/09948 PCr/US9 1/08948 

1/2 


O 

tl 


II 

CL 


II 
> 


OISJ 

o 


Eh 


II 


E 
o 
Z 


09 


Is 


Hill 


EC 





















^9 ' 





□ 

inn 



(A 

a; 

O 
to 


Ml 


o 

OO 


Q. 

OO o o 


5: 


SUBSTITUTE SHEET 


wo 92/09948 


PCr/US91/08948 


^& 2. 


^^^^ 



Send Scripts 
Check Nomes 
Install Into Swivel 
Inherit mode 

Add Marker 
Delete Marker 

Messages 
Status 

Orientation Editor 

Zero World 
50% 
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